Docket No.: IIS- 110 

REMARKS 

In the Office action mailed 12/31/03, the abstract of the disclosure was objected to 
because of length and a wording choice on line 11. The abstract submitted herewith 
remedies these problems by correcting the length to less than 150 words, and removing 
the word "said" from line 1 1 . 

The disclosure is also objected to at pages 11 and 12, due to spelling errors in the 
Incorporation by Reference section. These errors have been corrected herein. 

Claims 4-5 are objected to because of wording that recites "software program for 
providing..." Applicant thanks the Examiner for the suggested corrections, which have 
been adopted herein. 

35 USC 101 Rejections 

Claims 1-3 and 5-7 were rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter. The Examiner stated in the Office action mailed 
12/31/03 that claims 1-3 and 5-7 are not explicitly or implicitly declared to produce 
tangible and concrete results. Applicants request cancellation of claims 1-4 herein, and 
submits new claims 13-16, directed to a computer readable medium having instructions 
directed to control of a wireless transmitter. Likewise, claim 5 has been amended to be 
directed to control of a wireless transmitter. As claims 6-7 depend from claim 5, 
Applicant respectfully asserts that claims 5-7 as well as claims 13-16 are now in 
condition for allowance. 
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35 USC 103(a) Rejections 

Claims 1-3, 5-7, and 9-11 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Yamakawa et al. (U.S. Patent 6,633,858) in view of Konda et al (IDS reference 27). 
Applicant respectfully asserts that Yamakawa is irrelevant as it deals with discrete Q 
learning. For example, Yamakawa uses expressions like hash tables, vector lists, and 
action lists. All are meant to deal with discrete Q learning something that well known in 
the art. However, the present invention is directed to reinforcement learning applied to 
fuzzy logic rule-based control. 

In the Examiner's rejection of our claim 4 on page 15 line 1 1, the examiner references 
Sutton's work on wireless transmitters. However, Sutton does not use a fuzzy rule base 
to instruct the control strategy. 

The Examiner cites Yamakawa at Fig.19, col.15, lines 27-35 as disclosing "a control 
program. ..read a control program." In summary, this portion of Yamakawa states that 
there is a control program that directs a computer to execute a generation process for an 
action sequence performed by each of problem solvers according to a set of 
embodiments stored to a computer readable storage medium. The program does what 
Yamakawa is asking it to do. But the problem is with Yamakawa's method, which does 
not approximate the present invention. Yamakawa restricts his method to discrete 
states. This is much simpler problem to solve than continuous problem solving method 
that the present invention introduces. In order to extend discrete reinforcement learning 
to continuous reinforcement learning, Applicants have used a fuzzy rule base. There is 
not even single reference to a fuzzy rule base in the whole text of Yamakawa. 

The Examiner cites Yamakawa at col.2, lines 1-7 as disclosing "[a]n actor-critic 
model. . .an actor module." This paragraph only repeats a well-known fact of 
reinforcement learning. Yamakawa's approach uses it for a simple discrete case of 
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reinforcement learning. The approach can only work for problems with a limited number 
* of states and definitely not for continuous reinforcement learning with infinite number 
of states as the present invention does. 

The Examiner cites Yamakawa at col. 2, lines 36-40 and FIG. 3 as disclosing "a database 
of fuzzy-logic rules for mapping input data to output commands for mapping a system 
state." However, there is no mention of fuzzy- logic rules anywhere in this portion of 
Yamakawa nor anywhere else in Yamakawa. Thus, the present invention is completely 
different from Yamakawa's method and Yamakawa 5 s method cannot solve the problems 
solved by the present invention in the form of continuous reinforcement learning where 
there are a large number of states. 

The Examiner cites Yamakawa at Fig. 17; col. 11, lines 18-28 as describing the moveable 
state... movable state list." This paragraph is another distinguishing statement about 
the limited scope of Yamakawa's approach when compared to the present invention. 
Again, this portion of Yamakawa deals with discrete reinforcement learning, but cannot 
be extended to continuous reinforcement learning with infinite number of states. 
Again, the present invention can solve continuous reinforcement learning because it 
uses a fuzzy rule base. Yamakawa's approach does not use a fuzzy rule base. 

Similarly, Yamakawa at col. 12, lines. 13-17 cannot be said to teach, disclose, nor 
otherwise suggest continuous reinforcement learning. 

Yamakawa at col. 14, lines 41-45 refers to developing a landmark database composed of 
a segmentation function. This strategy cannot work in a continuous reinforcement 
learning scenario. So again, Yamakawa's method does not apply to continuous 
reinforcement learning. In contrast, the present invention utilizes a fuzzy logic rule base 
that provides the necessary function approximation. 
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Yamakawa at FIG. 16; col. 1 1, lines 18-53 describes his approach using expressions such 
as lists, hash tables, action vectors, etc. which only make sense if the number of states 
are limited. These statements can only be made in the context of discrete reinforcement 
learning case. Such statements cannot be made for continuous reinforcement learning, a 
claimed limitation of the present invention. 

While the Examiner claims that it would have been obvious to extend Yamakawa' s 
approach with what Konda teaches, Yamakawa's approach only works for discrete 
domains with limited states and actions. Thus, the claimed limitations of the present 
invention are not taught, disclosed, nor suggested by Yamakawa and Konda. 

Similar reasons are given to support the Examiner's rejection of claim 5. Thus, 
Applicant's arguments above are respectfully submitted in response to Examiner's 
rejection of claim 5. 

As for claim 9, Examiner asserts that Yamakawa discloses a system controlled by an 
actor-critic based fuzzy reinforcement learning. On the contrary, in the whole text of 
Yamakawa's patent there isn't even once use of the term "fuzzy" let alone a description 
of actor-critic based fuzzy reinforcement learning. 

Claims 4, 8, and 12 are rejected under 35 USC 103(a) as being unpatentable over Sutton 
et al. However, like Yamakawa and Konda, Sutton does not use a fuzzy rule base in his 
wireless transmitter's method. 

Furthermore, there is no suggestion to combine the references cited by the Examiner to 
achieve an actor-critic-based fuzzy reinforcement learning and apply it to a wireless 
transmitter's power control. 
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• Applicant respectfully submits that all of the rejections have been traversed requests 
the examiner place the present application in condition for allowance. 



Date: April 30, 2003 
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